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Abstract 

For the last few years, we assist to a growing interest of designing 
AQM (Active Queue Management) using control theory. In this paper, 
we focus on the synthesis of an AQM based on the Lyapunov theory 
for time delay systems. With the help of a recently developed Lyapunov- 
Krasovskii functional and using a state space representation of a linearized 
fluid model of TCP, two robust AQMs stabilizing the TCP model are 
constructed. Notice that our results are constructive and the synthesis 
problem is reduced to a convex optimization scheme expressed in terms 
of linear matrix inequalities (LMIs) . Finally, an example extracted from 
the literature and simulations via NS simulator ^ support our study. 

Keywords: Active Queue Management, congestion problem, Linear 
time delay systems, Lyapunov-Krasovskii functional, LMIs. 

1 Introduction 

Over a past few years, problems have arisen with regard to Quality of Service 
(QoS) issues in Internet traffic congestion control [TS', ^53]. AQM mechanism, 
which supports the end-to-end congestion control mechanism of Transmission 
Control Protocol (TCP), has been actively studied by many researchers. AQM 
controls the queue length of a router by actively dropping packets. Various 
mechanisms have been proposed in the literature such as Random Early De- 
tection (RED) [B], Random Early Marking (REM) P, Adaptive Virtual Queue 
(AVQ) [12] and many others [21]. Their performances have been evaluated [5], 
[3T] and empirical studies have shown the effectiveness of these algorithms [14| . 
Then, significant research has been devoted to the use of control theory to de- 
velop more efficient AQMs. Using dynamical model developed by [T7], some 
P (Proportional), PI [Proportional Integral) have been designed as well as 
robust control framework issued [20]. Nevertheless, most of these papers do not 
take into account the delay and ensure the stability in closed loop for all delays 
which could be conservative in practice. 
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The study of congestion problem with time delay systems framework is not 
new and has been succesfully exploited. In [TB], [TH], using Lyapunov-Krasovskii 
theory, the global stability analysis of the non linear model of TCP is performed. 
In [TT], a delay dependent state feedback controller is provided by compensa- 
tion of the delay with a memory feedback control. This latter methodology is 
interesting in theory but hardly suitable in practice. 

Based on a recently developed Lyapunov functional for time delay systems, 
two AQMs stabilizing the TCP model are constructed. The first one is called 
lOD-AQM {Independent Of Delay) and it deals with the robust control of TCP 
for all delays in the loop. The second one, DD-AQM {Delay Dependent) is de- 
voted to the control of the TCP dynamics when an upperbound of the delay 
is known. In order to consider a more realistic case, extension to the robust 
case, where the delay is uncertain is considered using quadratic stabilization 
framework. 

The paper is organized as follows. The second part presents the uncertain 
mathematical model of a network supporting TCP. Section III is dedicated to 
the design of two AQMs ensuring the robust stabilization of TCP. Section IV 
presents application of the exposed theory and the simulation results using NS- 
2. Finally, section V concludes the paper. 

Notations: For two symmetric matrices, A and B, A > {>) B means that 
A — B \s (semi-) positive definite. A^ denotes the transpose of A. !„ and Omxn 
denote respectively the identity matrix of size n and null matrix of size m x n. 
If the context allows it, the dimensions of these matrices are often omitted. For 
a given matrix Ae R"^", < A> stands for A + A^. 

2 Problem statement 

2.1 The linearized fluid-flow model of TCP 

The fluid flow model of TCP considered here was introduced in [T7] , [TU] . Based 
on this system, we will construct two AQM, which take into account delays 
inherent to networks. 

Given the network parameters: number of TCP sessions, link capacity and 
propagation delay {N, C and Tp respectively), we define the set of operating 
points (Wo, qo,Po) hy W — and q — 0: 



where W{t) is the congestion window, q{t) is the queue length at the con- 
gested router and R{t) is the Round Trip Time (RTT) which represents the 
delay in TCP dynamics, xq denotes the value of the variable x at the equilib- 
rium point. 

Assuming N{t) = N and R{t) = Rq as constants, the dynamic model of 
TCP can be approximated, around an equilibrium point, by the linear time 
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delay system [T7]: 



-T^{Sqit)-Sqit-hit)))-^Sp{t-h{t)) (2) 
6q{t) = ^SW{t) - ^Sq{t) 

where 6W = W — Wq , Sq ^ q — qq and Sp ^ p — po are the state variables and 
input perturbations around the operating point. The model ^ is valid only if 
the variations of these new variables are kept enough small. 

The input of our model ^ corresponds to the drop probability of a packet. 
This probability is fixed by the AQM. This latter has for objective to regulate 
the queue size at the router. 

For synthesis problem (see section[3]), we consider a state feedback. So that, 
the queue management strategy of the drop probability will be expressed as 

pit)=po + kiSWit) + k25q{t). (3) 

Remark 1 

i) It is possible to design a state feedback as it corresponds to a PD (Propor- 
tional Derivative) control law Jllf - Furthermore, although W(t) is not measured, 
one can estimate y{t) — the aggregate flow at the link U If . JlSf . 

ii) The main difficulty in all representations of TCP behavior is the exact 
estimation of network parameters (and not the state feedback control law). Two 
techniques are used: 

• Active measurements U3]j . consist in generating probe traffic in the net- 
work, and then observing the impact of network components and protocols on 
traffic: loss rate, delays, RTT, capacity... Therefore, as active measurement 
tools generate traffic in the network (intrusiveness) , one of their major draw- 
hacks is related to the disturbance introduced by the probe traffic which can make 
the network QoS change, and thus provide erroneous measures U3f . Sometimes, 
active probing traffic can be seen as denial of service attacks (DoS), scanning, 
or something else but in any case as hacker acts. Probe traffic is then discarded, 
and its source can he blacklisted. 

• Passive measurements refer to the process of measuring a network, without 
generating or modifying any traffic on the network. Passive monitoring is done 
with the capture of traffic and estimate off line networks parameters: It 's still to 
be non intrusive ( good estimation of parameters ) but not reactive. The passive 
evaluation relies on DAG system cards [SJ that represent references for such 
kind of measurements. 

Passive and active measurements is still a growing interest because exact 
estimation of networks parameters still difficult since the heterogeneity of au- 
tonomous systems \13f . A future idea (early introduced in this paper) for this 
problem is to consider uncertainties for parameters: This solution allows to use 
robust control theory in sense of poly tops. 

2.2 Time delay system approach 

In this paper, we choose to model the dynamics of the queue and the congestion 
window as a time-delay system. Indeed, the delay is an intrinsic phenomenon in 



networks. Taking into account this characteristic, we expect to reflect as much 
as possible the TCP behavior, providing more relevant analysis and synthesis 
methods. 

The linearized TCP fluid model ^ can be rewritten as the following time 
delay system: 



i{t) = Ax{t) + Adx{t -h) + Bu{t - h) 

xo{e) = (t){e), with e e [-h,o] 
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Sp{t) the input. 



where x{t) — [6W{t) dq'^{t)] is the state vector and u{t) 
4>{0) is the initial condition. 

There are mainly three methods to study time delay system stability: analy- 
sis of the characteristic roots, robust approach and Lyapunov theory. The latter 
will be considered because it is an effective and practical method which provides 
LMl (Linear Matrix Inequalities ^ ) criteria. To analyze and control the system 
([4]), the Lyapunov-Krasovskii approach [9j is used which is an extension of the 
traditional Lyapunov theory. 

In the literature, few articles using time delay systems approach to model 
TCP dynamic already appeared. In [21], a delay dependent robust stability 
condition was proposed and the design of a state feedback was derived. How- 
ever, the criterion used is quite obsolete and thus conservative. Then, other 
papers design control laws based on predictor The predictive approach 

is an interesting method theoretically but not in practice, moreover the delay 
has to be known exactly. T6] and [18^ use time delay system approach too and 
propose global stability analysis of the linear model. However synthesis is not 
considered. 

In this paper, we aim at providing methods to control system ([4]) with dif- 
ferent objectives: giving conditions for the nominal or robust stabilization for 
lOD and DD cases. 



2.3 Polytopic uncertain model 

The state space representation shows that matrices A, Ad and B depend on 
network parameters. Especially, it depends on the RTT i?o, a significant pa- 
rameter, which is difficult to estimate in practice. For a more rigorous study, it 
could be interesting to take into account some uncertainty on the delay Rq. 
Let then rewrite system Q as following 

x{t) = A{Ro)x{t) + Ad{RQ)x{t -h) + B{Ro)u{t - h). (6) 

With the polytopic approach, the idea is to insure the stability for a set 
of systems. Let suppose that Rq G [-Ro,„i„ , -Ro™„i.]i then matrices A, Ad and B 
belong to a certain set 

^l^{[A,Ad,B] I i?o e [i?o„„„,i?o_J} 

and we aim at looking for an AQM (expressed in term of state feedback) which 
stabilizes system for all matrices belonging to fi. However, the parameter 



Rq doesn't appear linearly in the matrices A, Ad and B. So that, the set fl 
defined by the uncertainty is non convex. 

A common idea in robust control theory is to look for a polytopic set V 
which includes the set f2. Using convexity property, it is much more easy to 
test the stability in closed loop for the overall polytop. If the stability of V is 
proved, then the stability of O is insured. 

In order to create the polytop P, we pose pi — P2 = -ki and — 
Rq. Since there are three uncertain parameters, the polytop will have riuj = 
8 vertices. For a bounded value Rq, the new uncertain parameters pi, \fi = 
{1,2,3} are bounded. So, the matrices of the uncertain system © are defined 
as 
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P3B0. 



The set fl is contained in the set V, 



where the w*^'^ are the vertices of V. 
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3 Stabilization using time-delay system approach 

In the previous section, an uncertain model of the TCP / AQM dynamic has been 
designed. This section is devoted to the construction of robust AQM stabilizing 
a such model. The first approach proposes the construction of an independent 
of delay (lOD) controller using convex optimisation schemes (LMI). In a second 
part, we describe a delay dependent (DD) method which takes into account the 
size of the delay. Using an information on the delay, we expect a reduction of 
conservatism and then an improvment of results. 



3.1 Independent of delay AQM design 

The idea is to insure the stability in closed loop for all delays as it has been pro- 
posed in [TU] using frequential arguments and traditionnal control tools. Here, 
we propose to use the following well-known Lyapunov-Krasovskii functionnal: 

V{xt)=x^{t)Px{t)+ [ x^{6)Qx{6)de. (8) 

Jt-h 

where the matrices P and Q are symmetric and positive definite. The choice of 
this Lyapunov-Krasovskii functional implies the following proposition. 

Proposition 1 /P/ System ^ for u{t) — is asymptotically stable Mil > Q if 
there exist real symmetric matrices P > and Q, such that 



^^P + PA + Q VAd 
AlV Q 



<0 (9) 



is satisfied. 



Now, we construct the following memoryless state feedback 



u{t) = Kx{t), K e R"^^" (10) 

to control system ^ [K is a constant matrix gain). This controller corre- 
sponds to our AQM. 

Applying (fTO|) to ([1]), we get the closed- loop system 

{ ±{t)^ Ax{t)+Adx{t-h) . ^ 

\ xo{e) = 0(6*), with 6 e [-h,0] ^ ' 

with Ad^ Ad + BK. 

Then, the following synthesis criterion can be easily derived from Q and 

m- 

Proposition 2 // there exist symmetric matrices R > 0, S and a matrix Z G 
R'"^", such that 

'hA^ + AK + S AdR + BZ]^. 

KA^ + Z'^B^ -S J < " ^^^> 

then, system |7p is stable under the control law mU\) with the feedback gain 
K = ZR-^. 

This latter proposition provides an lOD-AQM, which stabilize @ for 
all delays /i e R+. 



3.2 Delay dependent AQM design 

In this subsection, our goal is to design a controller which takes into account 
the upperbound of the delay. The delay dependent case starts from a system 
stable without delays and looks for the maximal delay that preserves stability. 

Generally, all methods involve a Lyapunov functional, and more or less tight 
techniques to bound some cross terms and to transform system [S] . These choices 
of specific Lyapunov functionals and overbounding techniques are the origin of 
conservatism. In the present paper, we choose a recent Lyapunov-Krasovskii 
functional (O [7J: 
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ds 



V x{s-^-^h) J 



where P G S" is a positive definite matrix, Q G S''" and i? G S" arc two 
positive definite matrices, r > 1 is an integer corresponding to the discretization 
step. Using this functional, we propose the following. 



Proposition 3 If there exist symmetric positive definite matrices P, R E R"^" , 
Q € R''"'*''"^ a matrix X £ R(r+2)nxn^ ^ scalar hm > 0, an integer r > 1 and a 
matrix K € R"^^" such that 



r + X5 + s-^x^ < 
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and S = -1 A Onx(r-l)n Ad ^ 

then, system (7p can be stabilized for all h < h„i by the control law u(t) = Kx{t). 
Proof 1 It is always possible to rewrite ill]) as = where 
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Using the extended variable ^{t) il6]) . the derivative of V along the trajec- 
tories of system 0) leads to: 
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C <0 



such that 
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where T G S'-''^^-'" depends on P, R, Q and the delay h. 
Using projection lemma '22 1, there exists X G p(r+2)nxn ^jj^/j ^/j^^ dig)] 
equivalent to Jj^P - 



Remark 2 



• There exists another equivalent form of this LMI in term of analysis (i.e. 
with K = 0) provided in and based on robust control tools. 

• In term of analysis, it is shown in '81 that for r — 1, this proposed function 
il3\) is equivalent to the main classical results of the literature. Moreover, 
in the same study it is proved that for r > 1 conservatism is reduced. 

Nevertheless, applying a state feedback (fTU)) . we have Ad ^ Ad + BK with 
the controller gain K appearing as a decision variable. Then, the condition 
becomes a BMI. That's the reason why in this paper, we propose a relaxation 
algorithm. The algorithm principle consists to alternate analysis and synthesis 
steps. 

First let define the synthesis LMI: 

T+<X[-1 A ■■■ yld + BK]><0 (19) 

where K e Rf^x" x is the slack variable which has been fixed. 
By the same way, we define the analysis LMI: 

r+<X[-l A ■■■ Ad + BK]><0 (20) 

where K is fixed. Then, we propose the following algorithm. 
Algorithm: 

• Slack variable initialization, X — Xq 

1. We solve the synthesis optimization 

\ S.t. LMI (Ull) 
A matrix gain called Ki is derived. 

2. We solve the analysis optimization with K — Ki. 

{ Knax, = „ max {h^} 
\ s.t. LMI ^ 

The new slack variable is derived Xi. 

• We test if h"^^^.^^ = 

• if true, there is no improvement on the maximal size of the allowable delay: 
end of the algorithm. 

• if false, the process is reiterated to the step (1) with a new slack variable 
and upperbound of the delay. 

Remark 3 At the test step, one always has h'^iax — ^max ■ Consequently, 
throughout the progression of the algorithm the upperbound h^ can not regress. 

Notes that the main problem, which is common in relaxation methods, 
remains the initialization of slack variables. 



4 Application to TCP/AQM dynamics and val- 
idation through NS-2 



In this section, we are going first to consider the nominal system in order to 
expose the control principle. Then, we will extend our methods to the robust 
case. For a realistic case, it is essential to insure stability in spite of the delay 
uncertainty. 



4.1 Numerical example 



As a widely adopted numerical illustration extracted from [TU] , consider the case 
when qo = 175 packets, Tp = 0.2 second and C = 3750 packets/s (corresponds to 
a 15 Mb/s link with average packet size 500 bytes). Then, for a load of iV = 60 
TCP sessions, we have Wq = 15 packets, po — 0.008, Rq = 0.246 seconds. We 
obtain the following open loop system 



" 6W{t) ' 




6m 





-0.2644 
243.9024 
-0.2644 0.0044 




0.0044 
4.0650 
" SW{t 
5q{t 



' 5W{t) 
5q{t) 
'h{t)) 
h{t)) 



(21) 



Matrix A is Hurwitz and applying lOD proposition [1] we observe that the LMI 
© is feasible. So, we conclude (PT|) is lOD stable and system (PTt is stable for 
all /i G R^. However, in order to avoid congestion and to regulate the queue size 
at a desired level in spite of uncertainty on delay, an AQM has to be implanted. 



4.2 lOD/DD Synthesis 

4.2.1 Independent of delay method 

In the lOD case, for nominal system only, it turns out that the delayed term, 
which can be viewed as a disturbance, can be eliminated choosing fci and fc2 as: 

K = [fci fcz] such that <^ 2}v°2 ^^^^ 

Thus if A is Hurwitz and for a state feedback gain K defined as P^ . then 
the system is lOD stable. Since an lOD stabilizing gain K can always be 
found (in nominal case), this method provides a systematic technique for the 
algorithm initialization (for DD synthesis). 

Concerning the robustness issue, let consider that Rq is uncertain such that 
Romin < -Ro < Romax ' ^hls systcm will be stable if the polytop V (see section 
12. 3p is stabilized. Using the quadratic stability framework [2], we propose the 
following result. 

Proposition 4 The system 0) will he quadratically stabilized with the state 
feedback u(t) — Kx(t) \lh > and Vi?o G [Romin^ RomaA^ there exist symmet- 
ric matrices R > 0, S and a matrix Z G R'"^", such that 



r4'^^ + ZTb('')^ -S 



< (23) 



is satisfied for all i = {1, 2, 8}. The matrix gain is given 

A^^"^ and i?^*-' correspond to the matrices values on the vertices cj''-* 

J^) ^ = 1,2,. ..,8 (24) 

Taking numerical example (|2ip. i?o nominal value is 0.246 seconds. The 
objective is to stabilize the uncertain system ^ for given bounds (i?o™i„ and 
i?o„<j3.), and to maximize the stability domain. 

Applying lOD LMI condition of the proposition IH we obtain the results of 
table I. 





"max 


Gain K 


0.1 


0.4 


10-'' [-0.3709 0.0062] 


0.15 


0.83 


10-'i[-0.4729 0.0079] 



Table 1: lOD state feedback gains to stabilize a polytop 
We can observe, in that case, decreasing the lowerbound is more restrictive 
than increasing the upperbound for the optimization problem. Nevertheless, 
because of the unavoidable delay in networks (like propagation delays), it is 
useless to look for a very small lowerbound. 

4.2.2 Delay-dependent method 

Using the relaxation algorithm previously exposed and lOD gain (1^^ for the 
initialization, we get the following results of the table II for the robust delay 
dependent case where a common Lyapunov-Krasovskii functional is found for 
each vertice of the polytop. Compared to lOD results, we improve sligthly the 
set of admissible delays. 



r 




Gain K 


hm 


1 


[0.1,0.45] 


10-^[-0.589 0.0244] 


0.56 


1 


[0.1,0.5] 


10-^[-0.321 0.0204] 


0.48 


2 


[0.1,0.45] 


10-''[-0.575 0.0240] 


0.62 


2 


[0.1,0.5] 


10-^[-0.272 0.0193] 


0.52 



Table 2: DD state feedback gains to stabilize a polytop 

Remark 4 

• If Romai > ^m, then system (0) is just stable for Rq G [^Omi„,^m] since 
Rq is the RTT and corresponds to the delay. 

• As expected, we obtain better results for r = 2, since hmax is larger. 

Our results can be compared with results from [24 where a robust delay depen- 
dent stabilization is designed. In [23, the system in closed loop is shown to be 
robustly stable for Rq G [0,0.216] while the proposed criterion of proposition [3] 
robustly stabilises the system for Rq e [0.1,0.5]. 



4.3 Simulations 



We aim at proving the effectiveness of our method using NS-2 4 , a network 
simulator widely used in the communication community. Taking values from the 
previous numerical example, we apply the new AQM based on a state feedback 
(i.e a simple constant matrix gain K). The target queue length go is 175 packets 
while buffer size is 800. The average packet length is 500 bytes. The default 
transport protocol is TCP-New Reno without ECN marking. 

For the convenience of comparison, we adopt the same values and network 
configuration than |10j who design a PI controller [Proportional- Integral) . This 
PI is configured as follow, the coefficients a and b are fixed at 1.822e — 5 and 
1.816e — 5 respectively, the sampling frequency is 160IIz. 

In the figure [U we apply the gain K from the table II which ensures DD 
robust stability. We compare our result with PI AQM provided by [lU]. It 
appears that our control allows a faster response as well as a smaller overshoot. 
Simulations of perturbed system is reported in figures [2] and [3l In figure [21 




Figure 1: Time evolution of the queue length: comparison between PI and state 
feedback Gain K. 



we have increased the propagation delay by 20 ms. Even if the system converges 
to a different reference point (slightly lower) , the queue size is stable and quickly 
regulated. 



Figure 2: Time evolution of the queue length for gain K (calculated from DD 
robust stabilization) with a perturbation on the delay. 



For more important pertubations (on the delay i?o or number of sessions N) , 
the system in closed-loop is still stable but the steady state changes since we 
converge to a new equilibrium point. In figure [31 a gain K is calculated from DD 



robust stabilization with an external perturbation. The scenario is composed 
as follows: 7 additive sources (UDP protocol) send 1000 bytes packet length 
with a IMbytes/s throughput between t = 40s and t — 45s. With the DD 
robust controller, the response is perturbed. The closed-loop system converges 
to the same reference, the queue size is stable and quickly regulated when the 
perturbation disappeared. 
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Figure 3: Time evolution of the queue length for gain K (calculated from DD 
robust stabilization) with a perturbation as UDP traffic. 



5 Conclusion 

In this preliminary work, we have proposed the construction of robust AQMs 
for the congestion problem in communications networks. The developed AQMs 
have been established by using Lyapunov theory extended to delay systems 
and semi definite programming to solve the Linear Matrix Inequalities. Note 
that the proposed methods have been extended to the robust case where the 
delay in the loop is unknown. Finally, the AQMs have been validated using NS 
simulator. 
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